Papers with Neural models
ExtraPhrase: Efficient Data Augmentation for Abstractive Summarization (2022.naacl-srw)
Copied to clipboard
| Challenge: | Recent studies indicated that neural methods are governed by the scaling law for the amount of training data. |
| Approach: | They propose a low-cost strategy to augment training data for abstractive summarization tasks by extracting summarized text and paraphrasing it. |
| Outcome: | The proposed strategy outperforms back-translation and self-training and is more cost-efficient when training data is small. |
Boosting Dialog Response Generation (P19-1)
Copied to clipboard
| Challenge: | Neural models generate the most common and generic responses all the time . Empirical results show that our method can significantly improve the diversity of responses generated by sequence-to-sequence models. |
| Approach: | They propose an iterative training process and ensemble method based on boosting to improve the diversity of responses generated by neural models. |
| Outcome: | Empirical results show that the proposed method significantly improves diversity and relevance of responses generated by all models. |
Long-Distance Dependencies Don’t Have to Be Long: Simplifying through Provably (Approximately) Optimal Permutations (P19-2)
Copied to clipboard
| Challenge: | Neural models at the sentence level often need to model the interaction between words . however, there is no guarantee that the standard ordering of words is computationally efficient or optimal . |
| Approach: | They propose to use a dependency parse as a proxy for inter-word dependencies in a sentence to simplify the sentence with combinatorial objectives imposed on the sentence-parse pair. |
| Outcome: | The proposed model improves classification accuracy and reduces classification error by 2.0% over the previous state of the art. |
Enhanced Transformer Model for Data-to-Text Generation (D19-56)
Copied to clipboard
| Challenge: | Neural models have shown significant progress on data-to-text generation tasks . data- to-text models generate descriptive texts from non-linguistic structured data . |
| Approach: | They propose a new data-to-text generation model which learns content selection and summary generation in an end-to end fashion. |
| Outcome: | The proposed model outperforms current state-of-the-art models on content selection precision and content ordering metrics. |
A Margin-based Loss with Synthetic Negative Samples for Continuous-output Machine Translation (D19-56)
Copied to clipboard
| Challenge: | Existing methods for word embeddings generate faster training with fewer learnable parameters. |
| Approach: | They propose a novel margin-based loss that uses only predicted and target embeddings . they argue that the loss is more consistent and interpretable than other margin--based losses . |
| Outcome: | The proposed model is more consistent and interpretable than other margin-based losses. |
Coherent and Concise Radiology Report Generation via Context Specific Image Representations and Orthogonal Sentence States (2021.naacl-industry)
Copied to clipboard
| Challenge: | Neural models for text generation are often designed in an end-to-end fashion, limiting their practical usability in downstream applications. |
| Approach: | They propose a method to compute image representations specific to each sentential context and exploiting diverse sentence states to ensure topical continuity and content diversity of generated radiology reports. |
| Outcome: | The proposed method outperforms baselines on objective metrics and human evaluations by 18% and 29% respectively in the evaluation for informativeness and content ordering respectively. |
PARE: A Simple and Strong Baseline for Monolingual and Multilingual Distantly Supervised Relation Extraction (2022.acl-short)
Copied to clipboard
| Challenge: | Recent approaches to distantly supervised relation extraction (DS-RE) encode each sentence in an entity-pair bag separately. |
| Approach: | They propose a simple baseline approach where sentences of a bag are concatenated into a passage of sentences and encoded jointly using BERT. |
| Outcome: | The proposed approach outperforms state-of-the-art models in monolingual and multilingual datasets. |
Is My Model Using the Right Evidence? Systematic Probes for Examining Evidence-Based Tabular Reasoning (2022.tacl-1)
Copied to clipboard
| Challenge: | Existing models that claim to reason about evidence should avoid spurious patterns . tabular inputs are well-suited for the study—they admit systematic probes . |
| Approach: | They propose to use tabular data to test whether models can reason about evidence . they show that a RoBERTa-based model fails to reason on the following counts . |
| Outcome: | The proposed model fails to reason on tabular data on the following counts . the model is over-sensitive to annotation artifacts and ignores relevant parts of the evidence . |
Just One is Enough: An Existence-based Alignment Check for Robust Japanese Pronunciation Estimation (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Existence-based alignment has been used to detect pronunciation errors in Japanese NLP, but finding reliable attention heads remains challenging. |
| Approach: | They propose a method that detects and corrects pronunciation errors in Japanese by using beam search. |
| Outcome: | The proposed method reduces hallucinations and omissions and improves pronunciation estimation by over 2.5%. |
Improving Factual Consistency Between a Response and Persona Facts (2021.eacl-main)
Copied to clipboard
| Challenge: | Neural models for response generation produce responses that are semantically plausible but not necessarily factually consistent with persona facts. |
| Approach: | They propose to fine-tune these models by reinforcement learning and an efficient reward function that explicitly captures the consistency between a response and persona facts as well as semantic plausibility. |
| Outcome: | The proposed model improves the rate of responses that are factually consistent with persona facts over its supervised counterpart while maintaining the language quality of responses. |
Zero-shot Fact Verification by Claim Generation (2021.acl-short)
Copied to clipboard
| Challenge: | Existing methods for fact verification require large datasets, which can be expensive. |
| Approach: | They propose a framework for training a robust fact verification model by using automatically generated claims that can be supported, refuted, or unverifiable from evidence from Wikipedia. |
| Outcome: | The proposed framework reduces the demand for human-annotated training data and improves a model's F1 from 50% to 77%, equivalent in performance to 2K+ manually-curated examples. |
RNN Architecture Learning with Sparse Regularization (D19-1)
Copied to clipboard
| Challenge: | State-of-the-art NLP models require hundreds of millions and even billions of parameters to perform, which can lead to memory usage and increased runtime. |
| Approach: | They propose a structure learning method that uses group lasso to learn sparse, parameter-efficient NLP models by pruning more than 90% of the weights of rational RNNs. |
| Outcome: | The proposed method learns sparse, parameter-efficient models without sacrificing performance relative to parameter-rich baselines. |
Evaluating the Tradeoff Between Abstractiveness and Factuality in Abstractive Summarization (2023.findings-eacl)
Copied to clipboard
| Challenge: | Abstractive summarization models generate fluent and well-formed output but lack semantic faithfulness, or factuality, with respect to the input documents. |
| Approach: | They propose new factuality metrics that adjust for the degree of abstractiveness . they propose to visualize the rates of change in factual as we gradually increase abstractiveity . |
| Outcome: | The proposed models generate fluent and well-formed summaries but lack semantic faithfulness, or factuality, with respect to the input documents. |
Minimal Supervision for Morphological Inflection (2021.emnlp-main)
Copied to clipboard
| Challenge: | Neural models for morphological reinflection tasks have proved to be extremely accurate given ample labeled data, yet labele d data may be slow and costly to obtain. |
| Approach: | They exploit orthographic and semantic regularities in morphological systems to exploit the orthographic regularities on their own to achieve respectable accuracy. |
| Outcome: | The bootstrapping method outperforms hallucination-based methods for morphological reinflection tasks. |
Efficient and Robust Question Answering from Minimal Context over Documents (P18-1)
Copied to clipboard
| Challenge: | Recent work shows that neural QA models are sensitive to adversarial inputs. |
| Approach: | They propose a sentence selector to select the minimal set of sentences to feed into a QA model. |
| Outcome: | The proposed system reduces training time and inference time by up to 13 times . it is comparable to or better than the state-of-the-art on SQuAD, NewsQA, TriviaQA and SQu AD-Open . |
Database reasoning over text (2021.acl-long)
Copied to clipboard
| Challenge: | Existing models cannot handle database queries such as “List/Count all female athletes who were born in 20th century”. |
| Approach: | They propose a modular architecture to answer database-style queries over multiple spans from text and aggregate them at scale. |
| Outcome: | The proposed architecture scales to databases containing thousands of facts whereas current models are limited by how many facts can be encoded. |
Robust Machine Reading Comprehension by Learning Soft labels (2020.coling-main)
Copied to clipboard
| Challenge: | Neural models have achieved great success on the task of machine reading comprehension, which are typically trained on hard labels. |
| Approach: | They propose a robust training method for machine reading comprehension models to address label sparseness problem by using three strategies to train models on soft labels. |
| Outcome: | The proposed method improves the baseline model performance and achieves state-of-the-art performance on NewsQA and QUOREF. |
Interpretability for Morphological Inflection: from Character-level Predictions to Subword-level Rules (2021.eacl-main)
Copied to clipboard
| Challenge: | Neural models for morphological inflection have recently attained very high results, but their interpretation remains challenging. |
| Approach: | They propose a linguistically-motivated variant to the encoder-decoder model with attention that incorporates a character-level cross-attention mechanism and a self-attention module over substrings of the input. |
| Outcome: | The proposed model performs well on three typologically-different languages and is highly interpretable. |
Learning from Context or Names? An Empirical Study on Neural Relation Extraction (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing datasets may leak shallow heuristics via entity mentions, thus contributing to the high performance on RE benchmarks. |
| Approach: | They propose an entity-masked contrastive framework for relation extraction to gain a deeper understanding on textual context and type information while avoiding rote memorization of entities. |
| Outcome: | The proposed framework improves the effectiveness and robustness of neural models in different RE scenarios. |
Trigger-Argument based Explanation for Event Detection (2023.findings-acl)
Copied to clipboard
| Challenge: | Existing works on ED use words or phrases to explain models’ inner mechanisms, but for ED, the event structure is more enlightening clues to explain model behaviors. |
| Approach: | They propose a Trigger-Argument based Explanation method which can utilize event structure knowledge to uncover a faithful interpretation for existing ED models at neuron level. |
| Outcome: | The proposed method can reveal the process by which the model predicts on the large-scale MAVEN and the widely-used ACE 2005 datasets. |
A Self-Training Method for Machine Reading Comprehension with Soft Evidence Extraction (2020.acl-main)
Copied to clipboard
| Challenge: | Existing models for machine reading comprehension lack evidence labels for training models. |
| Approach: | They propose a method which supervises the evidence extractor with auto-generated evidence labels in an iterative process. |
| Outcome: | The proposed method improves on three MRC tasks on seven datasets. |
Leveraging Structured Information for Explainable Multi-hop Question Answering and Reasoning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Neural models, including large language models (LLMs), achieve superior performance on multi-hop question-answering tasks. |
| Approach: | They propose to use the chain-of-thought mechanism to generate both the reasoning chain and the answer. |
| Outcome: | Empirical results show that the proposed framework generates more faithful reasoning chains and significantly improves the QA performance on two benchmark datasets. |
Latent Variable Sentiment Grammar (P19-1)
Copied to clipboard
| Challenge: | Existing neural models do not explicitly model sentiment composition, which requires to encode sentiment class labels. |
| Approach: | They propose a sentiment grammar that captures sentiment subtype expressions by latent variables and Gaussian mixture vectors. |
| Outcome: | The proposed model outperforms vanilla neural encoders on the Stanford Sentiment Treebank benchmark. |
Towards Semi-Supervised Learning for Deep Semantic Role Labeling (D18-1)
Copied to clipboard
| Challenge: | Existing methods for semantic role labeling require an immense amount of semantic-role corpora and are therefore not suitable for low-resource languages or domains. |
| Approach: | They propose a semi-supervised method that outperforms the state-of-the-art on SRL . method explicitly enforcs syntactic constraints by augmenting the training objective with a syntastic-inconsistency loss component. |
| Outcome: | The proposed method outperforms the state-of-the-art on limited SRL training corpora on CoNLL-2012 English section. |
Cognitive Alpha Mining via LLM-Driven Code-Based Evolution (2026.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to finding effective predictive signals from financial data are limited by their complexity and low signal-to-noise ratio. |
| Approach: | They propose a framework that combines code-level alpha representation with LLM-driven reasoning and evolutionary search. |
| Outcome: | The proposed framework combines code-level alpha representation with LLM-driven reasoning and evolutionary search. |
When and Why Does Bias Mitigation Work? (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Neural models exploit shallow surface features to perform language understanding tasks, rather than learning the deeper language understanding and reasoning skills that practitioners desire. |
| Approach: | They propose to use model debiasing techniques to pressure models away from spurious features and to use them to learn useful representations instead. |
| Outcome: | The proposed methods increase models' reliance on hidden biases instead of learning robust features that help them solve a task. |
Estimating Confidence of Predictions of Individual Classifiers and TheirEnsembles for the Genre Classification Task (2022.lrec-1)
Copied to clipboard
| Challenge: | Genre identification is a kind of non-topic text classification. genre is defined as a functional space. |
| Approach: | They propose to use SOTA to identify genres in non-topic texts . genres are functional and cannot be expressed just by some keywords . |
| Outcome: | The proposed models show that they perform better than their individual models in large datasets. |
Neuralizing Regular Expressions for Slot Filling (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods to integrate neural networks and symbolic rules have their merits and weaknesses. |
| Approach: | They propose to integrate regular expressions into neural networks for a slot filling task . they use finite-state transducers to convert regular expression into a neural network . their model has superior zero-shot and few-shot performance . |
| Outcome: | The proposed model outperforms rules in zero-shot and few-shot scenarios and is competitive when training data is available. |